fray (someone's) nerves

Translations
user les nerfs de quelqu’un
Definitions
To make someone feel very anxious, irritated, or stressed, often due to repeated annoyance or pressure.